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Abstract 


The  ZigBee  specification  provides  a  niche  capability,  extending  the  IEEE  802.15.4 
standard  to  provide  a  wireless  mesh  network  solution.  ZigBee-based  devices  require 
minimal  power  and  provide  a  relatively  long-distance,  inexpensive,  and  secure  means  of 
networking.  The  technology  is  heavily  utilized,  providing  energy  management.  Industrial 
Control  System  (ICS)  automation,  and  remote  monitoring  of  Critical  Infrastructure  (Cl) 
operations;  it  also  supports  application  in  military  and  civilian  health  care  sectors.  ZigBee 
networks  lack  security  below  the  “Network”  layer  of  the  Open  Systems  Interconnect  (OSI) 
model,  leaving  them  vulnerable  to  open-source  hacking  tools  that  allow  malicous  attacks 
such  as  Media  Access  Control  (MAC)  spoofing  or  Denial  of  Service  (DOS).  A  method 
known  as  Radio  Erequency  Distinct  Native  Attribute  (RE-DNA)  Eingerprinting  provides  an 
additional  level  of  security  at  the  Physical  (PHY)  level,  where  the  transmitted  waveform  of 
a  device  is  examined,  rather  than  its  bit-level  credentials  which  can  be  easily  manipulated. 
RE-DNA  fingerprinting  allows  a  unique  human-like  signature  for  a  device  to  be  obtained 
and  a  subsequent  decision  made  whether  to  grant  access  or  deny  entry  to  a  secure  network. 

Two  National  Instruments  (NI)  receivers  were  used  here  to  simultaneously  collect 
RE  emissions  from  six  Atmel  AT86RE230  transceivers.  The  time-domain  response  of 
each  device  was  used  to  extract  features  and  generate  unique  RE-DNA  fingerprints. 
These  fingeprints  were  used  to  perform  Device  Classification  using  two  discrimination 
processes  known  as  Multiple  Discriminant  Analysis,  Maximum  Eikelihood  (MDA/ME) 
and  Generalized  Relevance  Eearning  Vector  Quantization-Improved  (GREVQI).  Each 
process  (classifier)  was  used  to  examine  both  the  Eull-Dimensional  (ED)  and  reduced 
dimensional  feature- sets  for  the  high-value  PCI  Extension  for  Instrumentation  Express 
(PXIe)  and  low-value  Universal  Software  Radio  Peripheral  (USRP)  receivers.  The 
reduced  feature-sets  were  determined  using  Dimensional  Reduction  Analysis  (DRA)  for 
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both  quantitative  and  qualitative  subsets.  Additionally,  each  classifier  performed  Device 
Classification  using  a  “hybrid”  interleaved  set  of  fingerprints  from  both  receivers. 

The  FD  feature-set  used  for  Device  Classification  included  Nf=291  features.  When 
examining  each  single  receiver  seperately  and  averaging  over  both  classifiers,  FD  analysis 
achieved  an  arbitrary  benchmark  of  average  correct  classification  %C>90%  (cross-device 
average)  at  SNR^W.Q  dB  and  SNR~\6.Q  dB  for  PXIe  and  USRP  receivers  respectively. 
MDA/ML  performed  better  for  FD  feature  sets,  with  GRLVQI  requiring  SNR^l.Q  dB  in 
additional  gain  to  match  MDA/ML  performance.  DRA  was  used  to  evaluate  performance 
using  both  quantitatively  (Nf=5, 10, 33, 66, 99)  and  qualitatively  =99)  reduced  feature 
sets.  Quantitative  DRA  performance  favored  the  PXIe  receiver  which  consistently  achieved 
%C>90%  at  SNRxl2.0  dB  for  Af6[10  297].  Qualitative  DRA  showed  that  irrespective 
of  the  receiver  used,  the  Pfe-only  feature-set  outperformed  the  Fn^-only  and  Amp-only 
feature-sets.  Additionally,  when  using  FD  Nf =291  and  Nf=99  for  both  quantitative  and 
qualitative  feature-sets,  %Cfd  ~  %C<)9Qnt  ~  %C()<)Phz-  Finally,  when  developing  a  Hybrid 
Cross-Receiver  model  using  fingerprints  from  both  receivers,  testing  with  PXIe-only 
fingerprints  proved  to  be  the  most  effective  method  for  performing  Device  Classification. 
Both  classifiers  performed  better  than  in  any  other  hybrid  case,  achieving  the  %C>90% 
benchmark  for  Nf&[5  297]  and  6[10  297]  using  GRLVQI  and  MDA/ML,  respectively. 
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A  COMPARISON  OF  RF-DNA  FINGERPRINTING  USING  HIGH/LOW  VALUE 
RECEIVERS  WITH  ZIGBEE  DEVICES 


I.  Introduction 

This  chapter  provides  a  brief  introduction  to  the  operationally  motivated  research  of 
the  ZigBee  protocol  and  its  applications  in  Section  1.1.  Additionally,  Section  1.2 
provides  the  technical  motivation  for  using  the  Air  Eorce  Institute  of  Technology  (AEIT)’s 
Radio  Erequency  Distinct  Native  Attribute  (RE-DNA)  process.  Specifically,  Section  1.3 
provides  the  current  state  of  AEIT’s  RE-DNA  fingerprinting  process  based  on  previous 
research  [2-4,  6,  10,  11,  15,  20-23,  25-27,  29,  31-37,  39,  40,  45,  46]  and  advancements 
from  the  contributions  of  this  research  [30].  Einally,  a  breakdown  of  the  document 
organization  is  presented  in  Section  1.4. 

1.1  Operational  Motivation 

The  Institue  of  Electrical  and  Electronics  Engineers  (IEEE)  802.15  standard  provides 
guidance  for  establishing  a  Wireless  Personal  Area  Network  (WPAN)  [24].  WPAN  pro¬ 
vide  a  an  effective  way  for  anyone  to  establish  a  personal  network  to  which  a  multitude 
of  wireless  devices  can  be  connected  for  buisness  or  home  use.  The  ZigBee  specification 
provides  a  niche  capability,  quickly  growing  in  popularity,  within  this  standard  as  dictated 
by  IEEE  standard  802.15.4  [48].  ZigBee  provides  a  low  energy,  low  cost  alternative  that 
is  relatively  simple  to  set  up.  ZigBee  devices  also  boast  long  battery  life  and  the  ability  to 
perform  secure  networking.  Eor  these  reasons,  ZigBee  is  used  in  many  industries,  includ¬ 
ing  commercial,  military,  and  healthcare.  Commercial  industries  [13,  43,  44]  use  ZigBee 
for  Industrial  Control  System  (ICS)  and  building  automation,  energy  management  and  the 
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monitoring  of  critical  infrastructure.  The  military  has  begun  to  utilize  ZigBee  for  location 
and  positioning  [10]  while  medical  usage  includes  life-support  and  patient  monitoring  [5]. 
Given  the  strong  probability  that  personal,  critical,  and  even  military  national  security  in¬ 
formation  could  traverse  ZigBee  networks  on  a  day-to-day  basis,  the  requirement  for  strong 
network  security  remains  high. 

Wireless  networks,  such  as  ZigBee,  all  run  off  of  the  same  basic  premise  of  a  seven 
layer  model  known  as  the  Open  Systems  Interconnect  (OSI)  model ,  shown  in  Fig.  1.1  [1]. 
The  focus  for  security  tends  to  rely  soley  on  the  “Network”  and  “Data  Link”  layers.  This 
basic  security  is  where  a  network  relies  on  device  Media  Access  Control  (MAC)  or  Internet 
Protocol  (IP)  information  to  be  submitted  and  verified  as  authorized  prior  to  being  allowed 
into  a  network.  There  are  various  open  source  hacking  tools  that  detect  these  mechanisms 
by  spoofing  (replicating  and  presenting)  specific  device  bit-level  credentials  and  gaining 
unauthorized  network  access.  Once  inside,  a  malicous  device  can  perform  various  attacks 
such  as  denial  or  service,  network  key  sniffing,  or  hostile  takeover  of  the  whole  network. 
Of  these  hacking  tools,  there  are  a  few  that  specifically  target  vulnerabilities  of  ZigBee 
devices  including  KillerBee  [47]  and  Api-Do  [38]. 

This  increasing  threat  to  WPANs,  specifically  ZigBee,  has  constituted  establishing 
security  at  the  most  basic  “Physical”  level  of  the  OSI  model.  This  level  deals  with  the 
the  actual  physical  waveform  a  device  emits  when  it  transmits  or  receives  information. 
Ongoing  research  at  AFIT  has  attempted  to  establish  a  process  known  as  RF-DNA 
fingerprinting  whereby  a  specific  device  can  be  described  by  features  unique  only  to  it. 
A  network  can  then  use  this  prior-known  information  to  compare  to  a  device  requesting 
entry  into  the  secure  network,  assess  its  bit-level  credentials,  and  deem  it  as  authorized  or 
unauthorized  to  enter.  This  process  describes  each  device  with  a  unique  human-like  RF 
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signature  that  allows  for  highly  accurate  Device  Classification  and  makes  replication  of  or 
faking  identities  very  difficult.  It  is  for  this  reason  that  additional  RF-DNA  security  at  the 
“Physical”  layer  is  important  to  supplement  existing  “Network”  and  “Data  Link”  security, 
which  may  be  easier  to  bypass. 
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Figure  1.1:  Seven-layer  Open  Systems  Interconnect  (OSI)  Network  Model  [1]. 


1.2  Technical  Motivation 

This  RF-DNA  process  has  gone  through  multiple  generations  of  evolution  and 
currently  stands  as  shown  in  Fig.  1.2  [9].  Each  evolution  of  the  process  has  incorporated 
different  classification  and  verification  methods,  model  development  techniques,  as  well  as 
introduced  new  receivers,  signal  types,  and  feature-sets.  Each  of  these  new  or  modified 
methods  and  hardware  have  been  specifically  addressed  through  previous  work  at  AEIT 
[4,  6,  10,  11,  15,  20-23,  25-27,  29,  31-37,  39,  40,  45,  46],  with  the  aim  of  supplementing 
research  conducted  outside  of  AEIT  as  well  [2,  3,  6-8,  14-17].  Previous  work  specifically 
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on  ZigBee  has  gone  through  one  iteration  [9],  foeusing  mostly  on  Device  Verification  for  a 
single  reeeiver  generated  model  from  eolleetions  in  multiple  loeations.  The  researeh  here 
expands  upon  previous  ZigBee  work  with  the  introduetion  of  two  new  reeeivers,  one  of 
few  eurrent  efforts  to  provide  an  extensive  eomparison  of  elassifioation  methods,  and  a 
first-look  at  generating  a  Hybrid  Cross-Receiver  model  for  use  in  Device  Classification. 

1.3  Previous  vs.  Current  Research 

Table  1.1  provides  a  summary  of  teehnieal  areas  previously  addressed  in  developing 
AFIT’s  RF-DNA  proeess  [2-4,  6,  10,  11,  15,  20-23,  25-27,  29,  31-37,  39,  40,  45,  46]  and 
areas  addressed  by  this  researeh  [30]. 

1.4  Document  Organization 

The  remainder  of  this  doeument  is  organized  as  follows: 

Chapter  2  gives  basie  IEEE  802.15.4  ZigBee  signal  strueture  and  outlines  speeifie 
areas  of  interest  as  relevant  to  this  researeh.  Chapter  3  provides  the  researeh  methodol¬ 
ogy  used  to  implement  the  RE-DNA  fingerprinting  proeess  for  ZigBee  wireless  deviees. 
Speeifieally  diseusses  signal  eolleetion  using  a  high-value  and  low-value  reeeiver  as  well 
as  post-eolleetion  signal  proeessing  and  subsequent  RE-DNA  fingerprint  generation  using 
MATEAB.  Einally,  deseribes  the  Dimensional  Reduetion  Analysis  (DRA)  proeess  and  de¬ 
tails  deviee  diserimination  teehniques  utilizing  both  Multiple  Diseriminant  Analysis,  Max¬ 
imum  Eikelihood  (MDA/ME)  and  Generalized  Relevanee  Eearning  Veetor  Quantization- 
Improved  (GREVQI)  elassifieation  methods.  Chapter  4  provides  results  and  analysis  for 
both  full-dimensional  and  DRA  {qualitative  and  quantitative)  Device  Classification  perfor- 
manee  for  a  single -reeeiver  model.  Additionally,  a  eomparison  of  high-value  versus  low- 
value  reeeiver  performanee  is  provided  for  two  Device  Classification  methods,  MDA/ME 
and  GREVQI.  Einally,  results  and  analysis  is  provided  for  both  full-dimensional  and  quanti- 


4 


Figure  1.2:  AFIT  RF-DNA  Fingerprinting  Proeess  [9]. 
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tative  DRA  Device  Classification  using  a  Hybrid  Cross-Receiver  model  for  both  MDA/ML 
and  GRLVQI  processes.  The  document  concludes  with  Chapter  5  that  summarizes  re¬ 
search  activity,  highlights  significant  findings  and  provides  recommendations  for  follow-on 
research. 
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Table  1.1:  Technical  Areas  in  Previous  related  work  and  Current  research  contributions, 
“x”  denotes  areas  addressed. 


Technical  Area  Previous  Work  Current  Research 


Ref# 

Addressed 

Ref# 

ID  Time  Domain  (TD) 

[6,  15,  25,  26,  40,  45] 

[9-11,39,  40,  45,  46] 

X 

[30] 

ID  Spectral  Domain  (SD) 

[35,  46] 

2D  Wavelet  Domain  (WD) 

[25-27] 
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II.  Background 


This  chapter  contains  the  technical  background  that  serves  as  the  framework  for 
methodology  described  in  Chap.  3.  Section  2.1  describes  the  basic  ZigBee  signal 
structure  as  used  in  WPANs  under  IEEE  802.15.4.  Section  2.2  introduces  details  for 
Air  Eorce  Institute  of  Technology  (AEIT)’s  Radio  Erequency  Distinct  Native  Attribute 
(RE-DNA)  fingerprinting  process,  including  fingerprint  generation  and  calculation  of 
statistical  metrics  from  instantaneous  time-domain  signal  responses.  Section  2.3  describes 
elements  of  the  Multiple  Discriminant  Analysis,  Maximum  Eikelihood  (MDA/ME)  process 
and  Section  2.4  describes  the  Generalized  Relevance  Eearning  Vector  Quantization- 
Improved  (GREVQI)  process. 

2.1  ZigBee  Signal  Characteristics 

ZigBee-based  networks  adhere  to  guidelines  provided  in  IEEE  802.15.4  [24],  which 
specifies  structure  of  “Physical”  and  “Data-Eink”  (specifically  Media  Access  Control 
(MAC)-sublayer)  layers  for  ZigBee  device  transmission.  As  shown  in  Eig.2.1  [24],  ZigBee 
packets  exhibit  the  data  frame  specification  as  provided  in  standards.  The  MAC  sublayer 
contains  the  actual  transmission  including  the  associated  addressing  fields,  sequence 
numbers,  data  being  transmitted,  etc.  This  research  focuses  on  a  specific  Region  of 
Interest  (ROI)  for  exploitation  known  as  the  Synchronization  Header  Response  (SHR) 
as  shown  in  the  “Physical”  layer  in  Pig.2.1.  This  region  is  composed  of  a  5-octet,  two 
sequence  structure  as  shown  in  Pig. 2. 2.  The  SHR  includes: 

1.  Preamble:  A  4-octet  (32-bit)  binary  string  of  O’s  that  is  designed  to  provide  symbol 
chip  timing  for  the  transmitting  device. 

2.  Start-of-Prame-Delimiter  (SPD):  A  1 -octet  (8-bit)  binary  string  that  is  predefined  as 
(11100101).  It  is  designed  to  signify  the  end  of  the  preamble  and  thus  the 
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beginning  of  the  actual  transmission,  beginning  with  the  “Frame  Length”.  The  SFD 
is  known  alternatively  as  the  “sync”  frame,  which  is  used  throughout  this  paper. 

The  SHR  waveform  response  serves  as  the  ROI  for  this  research  because  of  its  defined 
standard  behavior,  which  is  supposed  to  be  identical  for  all  devices,  i.e.,  the  SHR  is  the  one 
part  of  any  ZigBee  transmission  that  remains  constant  [24].  Fully  independent  of  device 
type,  device  ID,  applications  being  performed,  etc.,  the  pre-defined  bits  and  corresponding 
waveform  response  remain  the  same  for  all  ZigBee  transmissions.  This  independence  is 
necessary  in  performing  later  model  development  and  classification  as  described  in  Chap. 
3.  Previous  research  [10]  failed  to  exploit  the  entire  SHR,  focusing  soley  on  the  “Preamble” 
for  developing  a  model  and  performing  classification.  After  further  analysis,  [9]  found 
that  exploitation  of  the  entire  SHR  (Preamble  +  SFD)  provided  notably  better  Device 
Classification  performance.  It  is  under  this  auspice  that  research  as  described  in  Chap. 
3  was  performed  and  subsequent  results  in  Chap.  4  reported. 
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Figure  2.1:  Physical  Layer  and  MAC  Sublayer  Structure  for  a  ZigBee  Packet  [24]. 
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Figure  2.2:  Physical  Protocol  Data  Unit  (PPDU)  packet  structure  for  IEEE  802.15.4  [24]. 
The  SHR  as  specified  on  the  left  is  the  ROI  for  this  research 


2.2  RF-Fingerprint  Generation 

An  RE-DNA  fingerprint  is  the  collective  term  used  to  describe  a  unique,  human-like 
signature  for  a  specific  wireless  device.  Each  fingerprint  is  generated  in  a  two-step  process 
that  includes  calculation  of  instantaneous  time  domain  signal  responses,  and  calculation  of 
statistical  metrics  of  those  signal  responses.  Each  step  is  further  discussed  below: 

2.2.1  Time  Domain  Signal  Responses. 

The  ZigBee  SHR  contains  a  unique  time-domain  waveform  in  which  instantaneous 
signal  responses  that  describe  it  can  be  calculated.  This  research,  as  described  and  exe¬ 
cuted  in  [9-11,  27,  30,  31,  36,  37,  40,  46]  focused  on  Nch  =  3  characteristic  instantaneous 
responses  (a  =  amplitude,  (p  =  phase,  /  =  frequency)  in  a  burst.  Each  collected  signal 
is  represented  as  complex  In-Phase  and  Quadrature  (I/Q)  component  pairs  which  both  re¬ 
ceivers  collect  and  store  in  the  form  of  16-bit  integers  [28]: 


[fo>  Qq,  h,  •••  Jnc^  Qnc\  ’ 

where  Nc  represents  the  total  number  of  collected  sample  I/Q  pairs.  The  corresponding 
instantaneous  time-domain  responses  {a,  0,/)  are  calculated  as  [30]: 


a[n\  =  +  QVnY, 


(2.1) 


10 


,  for  l{n\  0, 


(2.2) 


(p\n\  =  tan 


QVn] 

m 


d(p{n) 

dt 


(2.3) 


for  a  given  sample  number  n  =  1,2,3,...  ,  Nc. 

These  ealeulated  elements  of  the  SHR  are  then  normalized  [27,  40]  and  their  mean 
value  removed.  This  is  done  by  first  removing  the  mean  value  for  eaeh  element  within  a 
single  response  and  then  dividing  (normalizing)  the  colleetion  of  remaining  elements  by 
the  maximum  value.  This  is  accomplished  for  each  response  in  (2.1),  (2.2),  and  (2.3)  and 
yields: 


_  ,  ,  a[n]  -  fia 

adn)  =  ,  r  n,  ’ 

max{ac[n]] 

n 

(2.4) 

.  r  1 

<Pc\n\  =  ,  ,  r  T,  . 

max(0c[n]} 

n 

(2.5) 

-  _ 

msLx{fc[n]}  ■ 

(2.6) 

n 


where  (2.4),  (2.5),  (2.6)  show  the  respective  mean  (jUa,  and  /if)  being  removed  and 
“max”  notes  the  value  by  which  each  response  is  normalized;  these  are  the  normalized 
signal  responses  used  for  RF-DNA  fingerprint  generation. 

2.2.2  Statistical  Metrics. 

After  each  response  is  centered  and  normalized,  statistical  metrics  are  calculated. 
Following  [9-11,  27,  30,  31,  36,  37,  40,  46],  Nm  =  4  statistical  metrics  can  be  calculated 
(cr  =  standard  deviation  ,a^  =  variance,  y  =  skewness,  k  =  kurtosis)  for  each  response. 
This  is  done  by: 

1.  Dividing  the  SHR  into  Nr  equal  subregions  subject  to  the  constraint  that  Nc/Nr  is 
an  integer.  Additionally,  the  entire  SHR  is  examined  and  treated  as  a  region  itself, 
yielding  a  total  of  Nr+i  regions. 
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2.  Calculating  cr,  cr^,  y,  and  k  (as  selected)  for  each  response  sequence  adn),  (f)c[n],  and 
fc[n]  according  to 


1  ^ 

n=l 


(T  = 


=  A 

}  n=l 

-  iJ-f  , 

n=\ 

1  ^ 


n=\ 

N 


^{x{n\  -  ixf  , 


Ncr^ 


n=\ 


where  Nc  represents  the  total  number  of  collected  samples. 


(2.7) 

(2.8) 

(2.9) 

(2.10) 

(2.11) 


3.  Arranging  selected  (2.8)-(2.11)  metrics  in  a  vector  for  each  specific  region  as, 


pRi  =  [cTr,  0-%  Jr,  Kr^]ix4, 


(2.12) 


where  i  =  1, 2, 3, . . .  ,Nr+i.  An  example  of  this  process  is  shown  in  Fig.  2.3  [31]. 


In  total,  for  each  fingerprint  composed  of  Nch  instantaneous  responses  and  Nm 
statistical  metrics  per  response,  over  an  SHR  of  Nr+i  regions,  the  number  of  “full¬ 
dimensional”  (FD)  features  (Nfd)  is  calculated  as. 


Nfd  —  Nch^  Nm  x  Nr+i,  (2.13) 

where  each  full-dimensional  fingerprint  F  is  composed  of  the  calculated  statistics  for 
each  of  the  three  instantaneous  responses  and  shown  as 
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Figure  2.3:  Process  to  generate  a  unique  fingerprint  utilizing  statistical  metrics  for  each 
instantaneous  response  over  Nr+i  subregions  [31]. 


F  =  [F"  :  (2-14) 

These  full-dimensional  RF-DNA  fingerprints  are  then  used  for  model  development  and 
classification  for  all  devices  (classes)  using  the  two  methods  as  described  below. 

2.3  MDA/ML  Processing 

The  MDA/ML  model  development  and  classification  process  is  comprised  of  two 
seperate  processes  including  Multiple  Discriminant  Analysis  (MDA)  and  Maximum 
Likelihood  (ML)  estimation.  The  description  included  here  is  based  largely  upon  [9],  with 
selected  elements  included  here  for  completeness.  MDA  serves  as  a  method  to  develop  a 
model  utilzing  the  Training  RF-DNA  fingerprints  as  will  be  discussed  in  Chap.  3.  ML  is 
a  classification  method  that  uses  the  Testing  RF-DNA  fingerprints  to  compare  to  the  model 
generated  by  MDA  and  subsequently  perform  Device  Classification.  Both  processes  are 
presented  in  further  detail  below. 
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2.3. 1  Multiple  Discriminant  Analysis  ( MDA )  Model  Development. 

This  section  describes  how  a  model,  later  used  for  classification,  is  developed  using 
MDA.  MDA  is  a  process,  based  on  Fisher’s  Linear  Discriminant,  that  linearly  projects  a 
high-dimensional  data  space  into  a  lower-dimensional  one.  The  desired  effect  is  to  follow 
the  method  of  least-squares  whereby  a  generalized  linear  model  can  be  generated  [12]. 
Unlike  the  Fisher  method,  which  only  works  for  discrimination  of  a  Na  <  2  class  prob¬ 
lem,  MDA  works  to  reduce  the  feature  dimensionality  describing  an  RF-DNA  fingerprint 
through  projection  for  Nci  >  2.  MDA  takes  a  specified  number  of  feature  )-dimensional 
described  input,  and  projects  it  into  a  subspace  that  is  characterized  by  Nd  dimensions.  It 
is  noted  that  through  the  remainder  of  this  document,  the  term  “class”  is  interchangable 
with  a  single  “device”  and  accordingly  with  Noev  =  6  devices  as  described  in  Section  3.1, 
Nci  =  6  classes.  The  overall  goal  of  this  projection  is  to  maximize  the  distance  of  the  space 
describing  each  class  from  another  while  simultaneously  minimizing  the  spread  within  a 
class.  Mathematically,  this  directly  translates  to  a  desired  maximum  distance  between  the 
mean  of  each  class,  while  minimizing  the  variance  within  a  single  class  [12]. 

Two  scatter  matrices  required  for  MDA  are  the  out-of-class  (inter-class,  S^)  and  in- 
class  (intra-class,  S^)  matrices  [42].  These  two  matrices  are  used  to  assemble  the  required 
projection  matrix,  referred  to  as  W.  It  is  this  matrix  that  is  used  to  project  a  fingerprint  F, 
and  maintain  an  optimal  balance  ratio  between  inter-class  means  and  intra-class  variances 
as  described  in  [12].  These  scatter  matrices,  as  well  as  their  components  are  computed 
as  [42], 


Nci 

=  (2.15) 

i=l 

^ci 

S.,  =  YjP,(P:  -  Po)(Mi  -  Ilof  ,  (2.16) 

!=1 
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where  class  covariance  (E,)  and  the  global  mean  of  all  classes  ipio)  are  calculated  as 


=  (2.17) 

Nci 

Mo  =  ^PiMi.  (2.18) 

/=! 

yu,  and  Pi  as  referenced  in  (2.18)  are  the  mean  and  prior  probability  for  each  class 
respectively.  The  intra-class  scatter  matrix  in  (2.16)  provides  a  measure  of  the  sum  of 
probability-weighted  class  feature  variances  for  each  individual  class  while  the  inter-class 
scatter  matrix  in  (2.15)  provides  a  measure  of  the  average  distance  (over  all  of  the  classes 
combined)  between  individual  class  means  from  the  respective  calculated  global  mean  of 
all  classes  combined. 

The  -dimensional  input  RF-DNA  fingerprint  vectors,  shown  as  F  from  (2.14),  are 
then  projected  into  the  lower  (A£))-dimensional  subspace  using  the  projection  operator 
matrix,  shown  below  as 


f=W^F.  (2.19) 

1  ^ 

W  is  the  NpXNo  projection  matrix  formed  from  the  Nci-i  eigenvectors  of  S^,  and  f  is 
the  resulting  RF-DNA  fingerprint  after  projection  into  the  new  subspace  [42].  Each  of  these 
fingerprints  f  will  then  be  split  into  Training  and  Testing  sets  as  described  in  methodology 
in  Chap.  3.  An  example  of  MDA  projection  is  shown  in  Fig.  2.4.  Here,  Nci=3  classes 
are  represented,  resulting  in  a  Ad  =  2-dimensional  subspace.  Wi  and  W2  respectively 
represent  the  two  projection  matrices  described  above.  In  this  illustration,  following  the 
desired  maximum  mean  seperation  for  the  MDA  process,  Wi  represents  the  “best”  class 
separation  with  no  overlap  among  the  three  classes.  Once  the  best  projection  matrix 
(referred  to  as  the  actual  “model”)  has  been  determined  by  MDA,  and  all  Training  RF- 
DNA  fingerprints  describing  each  class  have  been  projected  onto  their  respective  subspaces. 
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model  creation  is  finished  and  the  process  of  Device  Classification  begins  using  ML  and 
the  projected  Testing  RF-DNA  fingerprints. 


Figure  2.4:  MDA  Projection  Representation  for  Nci=3  Classes  corresponding  to  projection 
onto  2-dimensional  subspaces  using  Wi  and  W2  [12]  operators;  Showing  maximum 
seperation  (no  overlap  among  the  different  classes),  Wi  is  the  optimal  projection  matrix 
(model)  in  this  case. 

2.3.2  Maximum  Likelihood  (ML)  Classification. 

ML  is  a  method  of  Device  Classification  that  uses  the  model  developed  with  MDA. 
As  previously  stated,  after  a  model  is  created  using  the  Training  RF-DNA  fingerprints,  ML 
takes  the  remaining  (and  unused)  fingerprints  describing  the  Testing  data  set,  and  performs 
Device  Classification.  ML  classification  begins  once  the  best  “model”  or  projection  matrix 
(W)  is  determined,  and  the  Testing  fingerprints  for  each  class  are  projected  onto  the 
subspace.  At  this  point  in  the  process,  the  Training  fingerprints  have  been  projected, 
and  the  mean  (fii),  and  covariance  (%)  for  each  individual  class  have  been  computed 
for  /=1, 2, . . .  ,Nci-  ML  operates  off  the  assumption  that  all  of  the  projected  data  is  a 
Multivariate  Guassian  (MVG)  distribution  and  hence  each  class  can  be  described  by  its  own 
class-dependent  //,  and  %.  Additionally,  the  ML  process  can  assume  that  the  covariance  for 
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each  class  is  identical  and  thus,  a  collective  estimated  covariance  describing  all  classes  can 
be  shown  as 


1  Na 

—  T 

Naii 


2,'. 


(2.20) 


With  the  assumption  of  each  class  as  a  MVG  distribution,  posterior  conditional 
probabilities  can  be  calculated  for  each  Testing  fingerprint  f  and  used  to  provide  a 
measurement  of  class  (c,)  likelihood.  Following  the  MVG  distribution  and  collective 
covariance  (2.20)  estimate,  likelihood  estimation  can  be  implemented  as  [31,  42], 


1 


(2;r)(^«-i)/2det(i:p) 


1/2 


exp(!r^) , 


(2.21) 


where  Te  is  calculated  as 


n  =  -^  (f  -  /),/  (f  -  ft)  .  (2.22) 

Ci  likelihood  values  as  used  for  ML  are  based  on  a  Bayesian  decision  theory.  Each  f  from 
the  Testing  data  set  is  assigned  to  a  specific  c,  by 

E(c,|f)>E(cy|f)  Vj^f.  (2.23) 

Again,  i=\,2, . . .  ,Nci  and  here,  is  known  as  the  the  conditional  posterior 

probability  that  a  given  f  belongs  to  a  specific  class  c,.  The  conditional  posterior  probability 
P  in  (2.23)  is  calculated  using  Bayes’  Rule  using  specific  c,  likelihood  values  as  shown 
below  [31,  42]: 
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(2.24) 


P 


P{i\c)P{ci) 


It  is  assumed  that  P(ci)=  l/Nci,  meaning  that  all  prior  probabilities  are  equal  for  all  classes. 
This,  coupled  with  the  fact  that  for  any  given  f  fingerprint,  is  the  same  for  all  c, 
as  applied  to  (2.24),  allows  for  simplication  when  making  a  comparison  using  (2.23). 
Classification  is  then  performed  on  a  single  Testing  f  using  criteria  in  (2.23).  Each  f 
is  assigned  a  specific  c,  “label”  based  on  maximum  posterior  probability.  A  “correct 
classification”  occurs  if  the  assigned  c,  label  matches  the  true  or  known  c,-  label.  This 
process  is  repeated  on  all  Testing  fingerprints. 

2.4  GRLVQI  Processing 

The  second  method  considered  for  model  development  and  classification  is  GRLVQI. 
The  description  included  here  is  based  largely  upon  [31],  with  selected  elements  included 
here  for  completeness.  Unlike  MDA/ML  which  is  a  two-stage  process  of  model 
development  and  classification,  GRLVQI  is  a  one-stage  process  that  develops  a  model 
and  performs  classification  simultaneously.  GRLVQI  provides  some  advantages  over 
MDA/ML,  including: 

1 .  No  required  assumption  of  MVG  distribution;  GRLVQI  does  not  require  knowledge 
of  or  assumption  of  any  specific  statistical  distribution. 

2.  Model  Development  and  Device  Classification  are  performed  jointly,  rather  than  as 
independent  processes. 

3.  Each  input  feature  is  assigned  a  relevance  value  (A)  that  allows  for  feature  ranking 
and  Dimensional  Reduction  Analysis  (DRA),  as  described  in  Section  3.4. 

The  model  generated  by  GRLVQI  utilizes  prototype  vectors  that  describe  a  specific 
space  for  each  class.  The  number  of  prototype  vectors  Np  is  pre-defined  before  model 
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development  and  classification  take  place.  Np  is  the  same  for  all  classes  and  each  prototype 
vector  used  is  comprised  of  Np  features.  The  collection  of  all  prototype  vectors  that 
describe  the  classes  is  represented  by  p",  and  given  by  [18] 

P"  =  [P]  {NcrNp)xNF^  (2.25) 


where  Na  is  the  number  of  classes  and  P  is  a  matrix  that  defines  the  classification 
boundaries  for  the  prototype  vectors.  The  overall  goal  is  to  minimize  Bayesian  risk  by 
iteratively  shifting  the  intra-class  (p")  and  inter-class  (p^)  prototype  vectors  that  describe 
the  space  for  all  classes  until  a  “best  fit  model”  is  achieved.  This  shift,  d",  is  computed 
as  [18] 


Nf 

=  ,  (2.26) 

/=! 

where  is  a  randomly  chosen  Training  input  fingerprint  to  start  the  process,  and  n  is  the 
prototype  vector  from  (2.25)  such  that  n  =  1, 2, 3, ... ,  Np.  Ai  is  also  randomly  chosen  when 
the  process  is  started. 

This  iterative  process  continues  [31]  until  the  prototype  vectors  are  arranged  in  a  best- 
fit  model  and  the  corresponding  d,  values  determined.  Each  d,  receives  a  ranking  (number) 
that  indicates  its  importance  in  classification;  each  i  is  known  as  an  “index  number”  and 
directly  corresponds  to  a  single  feature.  All  d,’s  are  organized  into  a  single  vector  known 
as  Ap  where  it  is  possible  for  different  index  numbers  (different  features)  to  have  the  same 
relevance  ranking  (d)  value,  however, 

Nf 

Z  2,-  =  dfi  =  1. 

(=1 
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The  higher  the  T,  value,  the  more  important  or  relevant  it  is  in  performing  Device 
Classification  as  will  later  be  diseussed  in  Seetion  3.4.  Finally,  the  Testing  fingerprints 
(f)  are  plaeed  in  2.26  one  at  a  time  aeeording  to  the  best  model,  and  the  euelidean-distanee, 
between  eaeh  f  and  the  prototype  veetors,  is  ealeulated  as  shown  in  Fig.  2.5  [31].  The 
elassifieation  (assignment  of  that  partieular  f  to  a  speeifie  c,)  follows  aeeording  to  [31] 

Q  :  min(J^(p,.  .,f))  (2.27) 


Figure  2.5:  GRLVQI  Projeetion  Representation  for  a  single  fingerprint  f  for  Nci=3 
Classes[31]. 
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III.  Methodology 


This  chapter  contains  the  methodology  utilized  while  conducting  this  research  in  order 
to  obtain  the  results  as  presented  in  Chap.  4.  Simultaneous  emissions  collections 
were  taken  by  both  receivers  for  a  single  device  at  a  time.  These  emissions,  stored  by 
the  receivers  as  basic  In-Phase  and  Quadrature  (I/Q)  components,  were  then  converted 
into  complex  values  for  ease  in  signal  processing.  MATLAB  was  then  used  to  compute 
instantaneous  responses  of  detected  bursts  over  the  entirety  of  the  Region  of  Interest 
(ROI)  and  used  to  detect  and  extract  “bursts.”  The  ROI,  or  ZigBee  Synchronization 
Header  Response  (SHR),  was  then  down-converted  to  base-band  and  filtered  with  a  8^^- 
order  Butterworth  filter  to  remove  background  channel  noise.  Simultaneously,  Additive 
White  Gaussian  Noise  (AWGN)  was  generated,  like-filtered,  and  added  to  the  filtered 
SHR  to  appropriately  power-scale  and  achieve  SNR£[0  24]  dB.  This  resulting  signal, 
comprised  of  the  sum  of  the  down-converted  and  filtered  SHR  and  AWGN,  was  then 
broken  into  Nr  subregions.  These  subregions  were  then  used  to  calculate  statistical 
metrics  based  off  of  the  instantaneous  time-domain  signal  responses.  These  are  statistical 
“features”  used  to  generate  a  unique  Radio  Frequency  Distinct  Native  Attribute  (RF-DNA) 
fingerprint  that  was  used  in  Device  Classification  using  both  Multiple  Discriminant 
Analysis,  Maximum  Likelihood  (MDA/ML)  and  Generalized  Relevance  Learning  Vector 
Quantization-Improved  (GRLVQI)  processes.  Section  4. 1  describes  the  setup  for  collecting 
emissions.  Section  4.2  discusses  all  post-signal  collection  processing,  including  burst 
detection,  filtering,  and  AWGN-aided  SNR  scaling.  Section  4.3  discusses  how  an  RF 
fingerprint  was  generated  for  a  full-dimensional  feature-set,  while  Section  4.4  discusses 
the  process  known  as  Dimensional  Reduction  Analysis  (DRA).  Finally,  Section  4.4 
discusses  device  discrimination,  using  both  MDA/ML  and  GRLVQI  model  generation  and 
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classification  techniques.  Aeeounting  for  speeifie  reeeivers  used  here,  AFIT’s  entire  RF- 
DNA  Fingerprinting  proeess  is  as  shown  in  Fig.  3.1  [9]. 


RF  Signal  Intercept  & 
Collection  System  (RFSICS) 


.Vlmel  R/l  MUlkk 


PXIe1085 

USRP2921 


Post-Collection 

Processing 

(INLATLAB) 


Burst 
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Extraction 

Generation 

Signal 
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SNR-Scaled 
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Verification 

Figure  3.1:  AFITs  Fingerprinting  Proeess  [9] 


3.1  Signal  Collection 

Two  reeeivers,  the  National  Instruments  (NI)  PCI  Extension  for  Instrumentation  Ex¬ 
press  (PXIe)-I085  and  Universal  Software  Radio  Peripheral  (USRP)-2921,  eaeh  with  a  16- 
bit  Analog-to-Digital  Converter  (A/D),  were  used  to  eolleet  RE  emissions  from  six  Atmel 
AT86RE230  KillerBee  ZigBee  transeeivers  transmitting  at  2.48  GHz  per  IEEE  802.15.4. 
Eaeh  deviee  (eolleetively  referred  to  heneeforth  as  Atmel  RZUSBstiek  or  seperately  as 
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Devi,  Dev2,...,Dev6)  was  placed  2.0m  away  from  and  in  direct  line-of-sight  (LOS)  of  each 
receiver  as  depicted  in  Fig.  3.2  [30].  To  present  the  “most  challenging”  scenario  possible 
for  Device  Discrimination  and  obtain  an  accurate  comparison  between  the  higher-value 
PXIe  («  $150K)  and  lower- value  USRP  («  $2K),  multiple  controls  for  the  collection  pro¬ 
cess  were  established,  including: 


2  inetvrs 


-M  IISRP.2921 


.Vtniel  Ry,rSH>lick 


Figure  3.2:  Setup  to  collect  Atmel  RZUSBstick  emissions  using  PXIe  and  USRP 
receivers  [30]. 


1 .  Emissions  collected  on  one  device  at  a  time 

2.  Simultaneous  collections  of  the  same  device  emissions  by  both  receivers 

3.  Possible  receiver  clock  presence  (Fig.  3.3)  affecting  collection  center  frequency 
addressed  with  3MHz  offset 

Accounting  for  all  of  these  factors,  emissions  (also  referred  to  as  bursts)  were  collected 
for  each  device  at  a  sample  rate  of  fs  =  20  Msps.  Fig.  3.4  shows  a  one-sided,  expanded 
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view  of  Fig.  3.3. 


Figure  3.3:  Normalized  PSD  of  Atmel  RZUSBstick  eolleetion  noting  elock  presenee  near 
/  =  0Hz. 


Figure  3.4:  Normalized  PSD  response  from  Fig.  3.3  with  3MHz  eenter  frequeney  offset. 
Colleetions  taken  at  fs  =  20  Msps. 
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3.2  Post-Collection  Processing 

Following  emission  collections  on  all  ZigBee  devices  using  both  receivers,  a  series 
of  post-collection  processing  steps  were  performed  using  MATLAB  before  RF-DNA 
fingerprint  generation.  Collected  bursts  were  first  put  through  an  amplitude-based 
detection  process,  with  bursts  meeting  specific  criteria  “extracted,”  and  those  not  discarded. 
Extracted  bursts  were  down-converted  (center  frequency  shifted  to/  =  0),  and  subsequently 
placed  through  a  baseband  filter  to  remove  background  noise.  Finally  AWGN  was 
generated,  like-filtered,  power-scaled,  and  added  to  the  bursts  to  achieve  a  desired 
SNR£[0  24]  dB.  Each  of  these  steps  followed  those  from  previous  work  [9,  30,  31],  and 
are  described  specific  to  this  research  next. 

3.2.1  Burst  Detection. 

As  discussed  in  Section  2.2.1,  collected  emissions  from  both  receivers  were  stored  as 
interleaved  I/Q  components  according  to  [28]: 

Uo,Qo,h,Qi,  ••;Inc^Qnc\'^  (3.1) 

where  Nc  is  the  total  number  of  collected  samples.  Eor  easier  processing  in  MATEAB, 
each  I/Q  pair  was  converted  into  its  corresponding  complex  format  as: 

[(Iq  +  jQo),  (h  +  jQi),  ■,  (Inc  +  JQnc)]-  (3-2) 

The  collected  bursts  were  then  put  through  an  amplitude-based  detection  process  to  extract 
usable  bursts  out  of  the  background  noise.  Bursts  that  met  specific  detection  criteria  were 
determined  as  suitable  to  subsequently  turn  into  fingerprints  for  later  Device  Classification, 
while  others  were  discarded.  Specific  requirements  for  detected  bursts  included  specific 
leading  (ti)  and  trailing  (tj)  edge  thresholds,  as  well  as  minimum  (Emm)  and  maximum 
(Tmox)  time  duration,  shown  in  Table.  3.1. 
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Table  3.1:  Amplitude-based  burst  detection  parameters  for  ZigBee  transmission  collec¬ 
tions. 


Parameter 

Variable 

Value 

Leading  Threshold 

h 

-6.0  dB 

Trailing  Threshold 

tr 

-6.0  dB 

Min  Duration 

T  Min 

425  fisec 

Max  Duration 

Tmux 

550  fisec 

The  detection  process  began  with  the  instantaneous  amplitude  response  («[«])  being 
calculated  according  to  (2.1).  These  values  for  a  specific  collection  (multiple  bursts)  were 
then  converted  into  dBv  using: 


=  201ogio(-^^).  (3.3) 

This  provides  a  direct  relation  between  amplitude  and  dB  so  that  the  largest  a\n\dBv  for 
a  specific  n  can  be  found  throughout  the  entire  collection.  A  normalized  peak  value  is 
then  established  such  that  all  other  bursts  in  the  collection  meeting  the  required  ti  and  tj 
thresholds  from  this  peak  are  retained  for  possible  extraction.  If  a  burst  has  met  this  re¬ 
quirement,  it  is  then  examined  for  duration  requirements.  Recalling  that  fs  =  20  Msps, 
and  At  =  1  //^  =  0.05  //sec,  a  direct  relation  between  Nc  samples  and  time  duration  can  be 
established.  The  burst  duration  is  then  calculated  according  to  its  leading  sample  («/,)  and 
trailing  sample  (n-r)  where 


^Dur  —  {^T  - 


(3.4) 
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T Min  iJ^Dur  *  ^0  Tmqx- 


(3.5) 


If  threshold  and  duration  requirements  are  met,  a  burst  is  retained  and  “extracted”  for  use 
in  classification;  if  it  does  not  meet  the  requirements  it  is  discarded. 

3.2.2  Down  Conversion  and  Filtering. 

After  burst  extraction  was  completed,  the  retained  bursts  were  subsequently  placed 
through  a  two-stage  process  where  each  was  down-converted  to  baseband  (/c  =  0)  and 
filtered.  Each  process  is  outlined  below: 

1.  MATLAB  was  used  to  down-convert  each  extracted  burst  to  baseband  using  its 
own  center  frequency  estimate  derived  from  a  gradient-based  frequency  estimation 
process  [9,  30]. 

2.  The  down-converted  burst  was  placed  through  a  baseband  filter  to  remove  back¬ 
ground  noise  and  minimize  fluctuations  in  Device  Classification.  This  was  done 
as  in  [9,  30],  using  a  8'^-order  Butterworth  filter  having  a  baseband  bandwidth  of 
Wbb  =  IMHz.  The  result  was  a  single  burst,  centered  at  =  0,  with  minimal  ef¬ 
fect  from  noise  outside  of  the  collected  IEEE  802.15.4  channel.  This  process  was 
repeated  for  all  extracted  bursts,  with  one  such  burst  shown  in  Eig.3.5  [30]. 

3.2.3  SNR  Scaling. 

Einally,  to  provide  a  desired  analysis  range  of  SNRe[0  24]  dB  (SNRa)  for  Device 
Classification,  AWGN  was  created  using  MATEAB  and  like-filtered  before  addition  to 
each  burst  to  form  the  collective  SHR  used  to  generate  fingerprints. 

The  collected  S  NR  (S  NRc)  is  a  function  of  both  power  in  the  collected  signal  (without 
background  noise)  (Pc),  and  the  power  in  the  background  noise  during  collection  (Pn)-  Pn 
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Figure  3.5:  Normalized  PSD  Response  for  Atmel  RZUSBstick  after  baseband  down- 
conversion  and  application  of  8'^-order  Butterworth  Filter  Wbb  =  I  MHz  [30]. 

was  measured  when  none  of  the  six  devices  was  transmitting  so  that  it  represents  a  true 
“noise-only”  power.  The  resulting  S  NRc  in  dB  is  represented  as 


SNRc  =  lOxlogio 


(3.6) 


which  for  typical  ZigBee  collections  here  ranged  from  S  NRc  ~  24  dB  (USRP)  to  S  NRc 
«  30  dB  (PXIe). 

The  total  signal  (srot)  used  for  analysis  over  SNRa  is  the  summation  of  the  received 
signal  with  no  noise  (Sr),  background  noise  alone  (sn),  and  added  AWGN  (sgn),  and  is 
given  by 


(3.7) 


Slot  -  Sr  +  s^f  +  Sgn  ■ 


The  average  power  in  sgn  required  to  achieve  SNRa  is  noted  as  Pgn-  The  noise  samples 
used  to  create  the  desired  average  AWGN  were  generated  using  a  random  sequence  with  a 
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normal  distribution  that  was  complex  with  zero  mean.  With  Pqjsi  =  1,  the  associated  scale 
factor  (S  f)  required  to  obtain  the  desired  analysis  S  NRa  is  given  by 


S  F  - 


V 


10- 


XPr 


(3.8) 


Following  the  definition  of  S  NR  as  the  ratio  of  total  signal  (without  noise)  to  total  noise 
(without  signal),  (3.6)  can  be  rewritten  to  incorporate  Pgn,  shown  as 


SNRa  =  10  X  logio(p  >  (3.9) 

\fN  +  rQFil 

where  it  is  noted  that  generally,  the  scaled  AWGN  power  is  far  greater  than  the  collected 
background  noise  power  (Pgn»Pn)-  This  allows  for  (3.9)  to  be  simplified,  reducing  it  to 


SNRa  =  lOxlogio(-^W)  .  (3.10) 

vgn! 

Finally,  the  total  estimated  average  power  for  Fgv  can  be  calculated  following  the 
expression  for  the  total  of  any  given  arbitrary  complex  sequence  as  shown  below 

^  Nc 

Pgn  =  —  F  •  nGN(i)S F  •  n*GFi{i) .  (3.11) 

tl" 

where  UGNii)  is  the  real  power,  and  is  the  complex  conjugate  or  reactive  power, 

over  /=1, 2, . . .  ,Nc  total  samples.  As  described  above,  this  process  of  generating  AWGN 
and  appropriately  scaling  it  such  that  Device  Classification  could  be  performed  over 
SNRa&[0  24]  dB,  was  repeated  and  subsequently  added  to  each  extracted  burst  before 
RF-DNA  fingerprint  generation.  Finally,  it  is  noted  that  S  NRa  henceforth  is  referred  to  as 
SNR£[0  24]  dB  throughout  this  document. 

3.3  RF  Fingerprint  Generation 

The  overall  ZigBee  SHR  as  described  in  Section  3.2.3  (SHR  +  AWGN),  while  similar 
in  structure  when  defined  in  terms  of  transmitted  bits,  is  slightly  different  for  each  device 
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in  terms  of  the  physical  waveform.  The  SHR  therefore  contains  each  device’s  unique 
signature.  This  unique  signature  comes  from  manufacturing  tolerances,  device  aging 
characteristics,  and  differences  in  the  manufacturing  process.  It  is  this  unique,  device¬ 
dependent  signature  that  is  exploited  to  generate  RF-DNA  fingerprints  and  perform  Device 
Classification. 

IEEE  802.15.4  defines  the  first  5  octets  for  all  ZigBee  signals.  The  first  Tp  =  128 
yusec  of  each  transmission  corresponds  to  the  preamble,  and  the  following  Tsyn  =  32  //sec 
contains  the  synchronization  information  [24].  The  collective  preamble  and  sync  informa¬ 
tion  (128  //sec  -1-32  //sec)  makes  up  the  total  TsHR  =  =  160  //sec)  SHR,  as  illustrated  in 
Fig. 3. 6.  Accordingly,  the  duration  of  collected  emissions,  given  At  =  1//^  =  0.05  //sec  in 
each  sample,  is  given  by: 


SHR  Duration:  Tpoi  160  //Sec 

A 


0  0.1  0.2  0.3  0.4  0.5  0.6  0.7  0.8  0.9 

Time  (mSec) 


Figure  3.6:  ZigBee  transmission  showing  the  SHR  (highlighted  in  red)  and  the  payload 
(highlighted  in  blue). 
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128jU5ec 

Npre  =  - =  2560 samples.  (3.12) 

.05psec 

3^U.S€C 

^syn  =  “iT? - =  6A0samples.  (3.13) 

.05psec 

Ns  HR  =  2560  +  640  =  SlOOsamples.  (3.14) 

Recalling  the  constraint  that  Nc/Nr  must  be  an  integer  per  Section  2.2.2,  the  SHR  was 
divided  into  Nr  =  32  equal  subregions  of  100  samples  each,  beginning  at  the  start  of  the 
transmission  (burst)  and  ending  after  the  last  sync  sample  as  depicted  in  Fig.  3.7. 

Each  subregion  of  100  samples  contained  three  instantaneous  signal  responses  (a,  (f>, 
f)  that  uniquely  described  that  subregion.  Each  instantaneous  response  was  described  by 
three  RF-DNA  statistics  (cr^,  y,  k).  Accordingly,  cr^,  y,  and  k  were  calculated  per  (2.9)- 
(2.11)  for  each  a,  (j),  and  /  for  each  of  the  Nr  =  32  subregions  as  well  as  over  the  entire 
SHR  for  one  final  region  such  that  the  total  number  of  regions  was  Nr^i  =  33.  Each  statistic 
calculation  represents  a  single  RF-DNA  “feature.”  It  is  these  specific  features  that  uniquely 
generated  the  RF-DNA  fingerprints  used  for  MDA/MF  and  GRFVQI  Device  Classification. 
In  the  case  that  all  features  were  used  to  describe  a  unique  fingerprint,  known  as  “full- 
dimensional”  (FD),  the  number  of  features  is  shown  as 

Nrd  =  X  (cT^,y,K)  X  (Nr^,)  =  Nr  =  297.  (3.15) 

3.4  Dimensional  Reduction  Analysis 

A  process  known  as  DRA  was  performed  to  reduce  the  number  of  features  contained 
within  each  RF-DNA  fingerprint  for  a  given  device.  The  overall  DRA  goal  is  to  effectively 
reduce  computational  time  and  complexity,  while  maintaining  a  desired  comparable 
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Sample  Number 


Figure  3.7:  Magnitude  response  of  a  single  burst  SHR  divided  in  to  Nr  =  32  subregions 
for  subsequent  RF-DNA  fingerprinting.  100  samples  are  represented  between  each  vertical 
red  dashed  line. 


classification  performance  regardless  of  the  receiver  used.  Two  types,  Quantitative  DRA 
and  Qualitative  DRA  are  discussed  next. 

3.4.1  Quantitative  DRA. 

The  first  method  of  DRA  is  enabled  through  GRLVQI  and  deals  with  the  actual 
number  of  features  Np.  Recall  from  Section  3.3  that  the  full-dimensional  set  of  features 
describing  an  RF-DNA  fingerprint  for  the  Atmel  RZUSBstick  is  Npo  =  Np  =  297 
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features.  This  research  followed  the  method  in  [9,  31]  to  iteratively  reduce  Nf  to  a  level 
that  maintains  the  %C=90%  performance  benchmark  as  will  be  described  in  Chap.  4. 
Quantitative  DRA  takes  the  GRLVQI  relevance  vector  Ab  from  Section  2.4  and  selects  a 
specified  number  of  salient  features  such  that  only  the  top-ranked  d,  values  are  retained  and 
used  to  represent  the  characteristic  “space”  of  a  particular  class.  These  values  are  relevance 
ranked,  meaning  that  the  top-ranked  (highest-valued  d,)  features  have  the  greatest  impact  on 
Device  Classification.  Quantitative  DRA  can  be  performed  using  any  desired  Nf  provided 
the  chosen  value  adheres  to 


Nfd<Nf>0.  (3.16) 

Selection  of  specific  values  used  for  Np  here  are  described  in  Chap.  4,  where  results  for 
Np  =  5, 10, 33, 66,  and  99  features  are  provided. 

3.4.2  Qualitative  DRA. 

The  other  method  of  DRA  selects  feature-sets  as  a  given  instantaneous  signal  response 
subset  as  described  in  Section  2.2.  Again  the  full-dimensional  feature-set  for  a  given  RF- 
DNA  fingerprint  included  Npo  =  Np  =  297  features.  These  features  were  composed 
of  statisticas  (cr^,y,K)  calculated  for  the  instantaneous  (a,(f>,f)  responses  of  a  given  burst. 
Given  three  responses  for  each  full-feature  RF-DNA  fingerprint, 

Npia)  =  =  Npif)  =  99,  (3.17) 

where  during  RF-DNA  fingerprint  generation,  response  features  were  organized  sequen¬ 
tially  in  RF-DNA  fingerprints  according  to 

F  =  [F(a);F(0):F(/)]  1x297,  (3.18) 

where  indices  in  F  are  given  by: 
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a  :  /  6  [1  99];  0  :  i  6  [100  198];/  :  i  6  [199  297], 


(3.19) 


Each  subset  was  analyzed  seperately,  meaning  that  in  one  ease  for  example,  /-only 
features  were  used.  In  each  case,  relevance  rankings  (1  -  99  with  “1”  being  noted  as  the 
“top  feature”  and  thus  having  the  biggest  effeet  on  Device  Classification)  were  assigned  to 
the  given  subset  of  d;  values.  This  process  was  repeated  for  eaeh  instantaneous  response 
(a,/,/)  to  obtain  an  aceurate  eomparison  among  the  subsets  of  their  effeets  on  Device 
Classification.  Additionally,  this  enabled  direet  eomparison  with  Quantitative  DRA  for 
Np  =  99  (where  the  number  of  features  may  be  composed  of  features  from  any  and  all  of 
the  a,  /,  or/subsets). 

3.5  Device  Discrimination 

As  described  in  Chap.  2,  two  methods  of  Device  Discrimination  were  performed 
using  identical  sets  of  RF-DNA  fingerprints,  MDA/ML  and  GRLVQI.  Fig.  3.8  [41] 
shows  the  basic  discrimination  process  (in  block  diagram  form)  that  all  classifiers  follow. 
Additionally,  DRA  was  performed  using  a  model  generated  from  GRFVQI  and  its 
associated  A  values;  all  methods  of  Device  Classification  performed  will  be  shown  in 
Section  3.5.3. 

3.5.1  MDA/ML  Model  Development  and  Classification. 

The  development  here  is  taken  exclusively  from  [9]  and  presented  here  for  complete¬ 
ness.  MDA/MF  as  described  in  Section  2.3  is  a  two-step  discrimination  process  that  in¬ 
volves  both  MDA  model  development  and  MF  Device  Classification.  It  is  an  extension 
of  Fisher’s  Finear  Discriminant  and  used  for  Na  >  2.  This  research  was  conducted 
for  six  devices  (classes),  Nuev  =  Na  =  6,  and  it  thus  follows  from  Section  2.3.1  that 
Nd  =  (Na  =  6)  -  1  =  5  dimensions. 
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Figure  3.8:  Block  diagram  of  device  discrimination  process  showing  model  development, 
usage  of  test  statistics,  and  subesequent  classification  [41].  It  is  noted  that  “verification”  is 
possible  after  model  development  as  well  but  not  addressed  in  this  research  and  therefore 
grayed  out. 


MDA/ML  was  performed  using  RF-DNA  fingerprints  from  both  the  PXIe  and  USRP 
NI  receivers  for  three  different  models: 


35 


1 .  Single  Receiver:  PXIe-only  Training  fingerprints 


2.  Single  Receiver:  USRP-only  Training  fingerprints 

3.  Hybrid  Cross-Receiver:  Combined  PXIe  and  USRP  Training  fingerprints 

Each  model  was  developed  by  MDA  through  input  feature  dimensional  reduction.  The 
full-dimensional  feature-set,  Nf=291,  RF-DNA  Training  fingerprints  were  projected  onto 
the  A£)=5-dimensional  subspace.  This  was  done  using  an  iterative  method  known  as 
a  X-fold  process  as  described  in  detail  in  [9].  X-fold  is  a  method  that  uses  cross- 
validation  to  develop  the  “best”  model  for  use  in  Device  Classification  as  shown  in  Fig.  3.9 
[9].  The  best  model,  as  discussed  in  Section  2.3.1,  refers  to  the  model  that  leaves  the 
maximum  distance  between  the  mean  of  each  class  and  simultaneously  minimizes  the 
variance  within  any  single  class  [12].  This  research  utilized  a  K=5  approach  for  model 
development.  Once  the  best  model  (VPe)  was  selected,  that  model  formed  the  projection 
matrix  (W)  as  discussed  in  Section  2.3.1.  The  fingerprints  (Fj)  were  then  projected 
according  to  (2.19),  with  the  resulting  f  representing  the  lowered-dimensional  projected 
RF-DNA  fingerprints.  Following  projection  of  all  RF-DNA  fingerprints,  the  feature  space 
describing  each  class  was  “mapped”  accordingly  into  the  Fisher  Space  (Fig. 3. 10)  such  that 
the  “decision  boundaries”  defining  each  respective  class  were  formed.  MF  was  then  used 
to  perform  Device  Classification. 

MF  classification,  as  described  in  Section  2.3.2,  was  accomplished  using  the 
remaining  Testing  fingerprints  that  were  set  aside  during  MDA  model  development.  Each 
of  the  Nuev  =  6  devices  were  represented  and  the  process  again  assumed  Multivariate 
Guassian  (MVG)  distributions.  The  distributions  were  each  described  by  their  class- 
specific  means  (/),)  where  i  =  1, 2,  ...(Na  =  6),  and  a  collective  covariance  for  all  devices 
(Ep)  as  calculated  in  (2.20).  Additionally,  all  prior  probabilities  and  device  likelihoods 
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were  assumed  to  be  equal.  Each  Testing  fingerprint  was  classified  one  at  a  time  following 
the  iterative  process  [9]: 

1.  Input  Testing  fingerprint  from  an  unknown  class  Cj. 

2.  Project  F,  into  the  Fisher  space  using  (2.19)  to  generate  projected  fingerprint  fy. 

3.  Associate  fy  to  one  of  the  known  classes  (devices)  based  on  its  maximum  conditional 
likelihood  probability  according  to 

Ci  :  argmax[  p(c;|fy)  ],  (3.20) 

i 

where  z=l,  2, . . .  ,  (Na  =  6)  and  p(c,jfy)  is  the  conditional  likelihood  probability  that 
projected  fingerprint  f,  belongs  to  class  c,.  The  overall  measure  of  effectiveness  for 
the  classifier  (%C)  is  the  percentage  of  the  time  the  classifier  correctly  assigns  the 
fingerprint  to  its  true  device  or  class  over  all  trials  performed.  “Correct”  classification 
notes  when  fy  is  classified  as  its  known  c,  for  a  single  trial. 

3.5.2  GRLVQI  Model  Development  and  Classification. 

The  development  here  is  taken  exclusively  from  [18,  31]  and  presented  here  for  com¬ 
pleteness.  GRLVQI  processing  follows  the  same  basic  process  shown  in  Fig.  3.8  [41]. 
Unlike  MDA/ML  though,  GRLVQI,  as  described  in  Section  2.4,  performs  model  develop¬ 
ment  and  Device  Classification  jointly,  rather  than  as  two  independent  processes. 

GRLVQI  requires  no  prior  assumption  of  MVG  distribution  and  additionally, 
requires  no  knowledge  of  or  assumption  of  any  specific  statistical  distribution  for  model 
development.  It  uses  a  specified  number  of  prototype  vectors  to  “shape”  the  space  of  each 
class.  This  research  utilized  Np  =  10  prototype  vectors  to  describe  each  class,  with  each 
prototype  vector  being  comprised  of  Np  =  297  features  for  full-dimensional  analysis.  The 
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collection  of  all  prototype  vectors,  p"  as  derived  in  (2.25)  [18],  was  used  to  iteratively  shift 
intra-class  (p”)  and  inter-class  (p°)  prototype  vectors  until  a  “best  model”  was  achieved 
according  to  (2.26)  [18].  Model  development  and  subsequent  classification  followed  the 
process  as  shown  below: 

1.  Randomly  choose  a  Training  fingerprint  (?”)  and  relevance-ranked  feature  (d,)  and 
input  to  (2.26). 

2.  Shift  prototype  vectors  describing  class  “space”  by  distortion  factor  J” 

3.  Continue  to  iteratively  shift  protype  vectors  by  and  update  corresponding 
relevance-rankings  (d,)  until  “best-fit  model”  is  achieved  by  defined  smallest 

as  described  in  [31] 

4.  Define  “best-fit”  relevance  ranking  vector  (As)  as  the  vector  containing  d,  for 

i  =  higher-valued  d,  correspond  to  most  relevant  features  used  in 

describing  a  class. 

5.  Measure  euclidian  distance  from  a  single  Testing  fingerprint  (f)  to  each  of  the 
prototype  vectors  as  defined  by  the  best  model. 

6.  Associate  the  unknown  f  to  one  of  the  known  c,  based  on  the  smallest  euclidean 
distance  to  the  prototype  vectors  of  a  specified  c,  following 


C;  :  min(J^(P;  .,f))  (3.21) 

i,j 

where  /=1,2, ...  ,  (Na  =  6)  and  j=l,2, ...  ,{Np  =  10).  As  with  MDA/ML, 
%C  provides  the  measurement  of  classifier  effectiveness  and  “correct”  classification 
occurs  when  is  classified  as  the  true  known  c,. 
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3.5.3  Comparative  Asses  merit  Test  Matrix. 

The  full  spectrum  of  assesments  performed,  including  full-dimensional  and  DRA 
(quantitative  and  qualitative)  using  both  MDA/ML  and  GRLVQI  is  shown  in  Table.  3.2. 
The  results  of  these  tests,  which  allow  for  an  accurate  comparison  of  both  classification 
methods,  as  well  both  receivers  are  further  discussed  in  Chap.  4. 


Table  3.2:  Comparative  Assesment  Test  Matrix:  1  Full-Dimensional  Baseline  (MDA/ML  & 
CRLVQI),  2  Quantitative  DRA  (GRLVQI),  3  Quantitative  vs.  Qualitative  DRA  (GRLVQI), 
and  X  denotes  Test  Not  Performed 


Mum  Feats 

MDAAIL 

GRLVQI 

PXIe 

USRP 

PXIe 

USRP 

Full  Dim 

297 

L2 

1.2 

1.2 

1.2 

Qual  DRA 

99 

X 

X 

3 

3 

Quan  DRA  99 

X 

X 

2 

2 

66 

X 

X 

2 

2 

33 

X 

X 

2 

2 

10 

X 

X 

2 

2 

5 

X 

X 

2 

2 
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Figure  3.9:  An  illustration  of  the  ^-fold  training  process  used  for  MDA  model 
development.  The  “best”  model  Wb  is  selected  based  on  the  Wk  that  yields  the  highest 
%Ck.  This  Wb  then  becomes  the  model  projection  matrix  used  for  subsequent  classification 
and  verification  [9] . 
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Figure  3.10:  Signal  collection,  post-collection  processes  and  .fiT-fold  cross-validation 
training  for  MDA  model  development.  This  depicts  a  representative  NDev='i  ZigBee 
devices  (classes)  and  the  corresponding  2D  Fisher  Space.  Each  (o)  represents  a  projected 
training  fingerprint  f  clustered  around  the  respective  class  means  shown  as  (•)  [9]. 
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IV.  Results  and  Analysis 


This  chapter  contains  device  discrimination  results  for  six  ZigBee  devices  using  emis¬ 
sions  collected  on  National  Instruments  (NI)  PCI  Extension  for  Instrumentation  Ex¬ 
press  (PXIe)  and  Universal  Software  Radio  Peripheral  (USRP)  receivers.  Device  Classi¬ 
fication  was  performed  on  both  full-dimensional  as  well  as  qualitative  and  quantitative 
Dimensional  Reduction  Analysis  (DRA)  feature-sets.  DRA  was  performed  using  Gen¬ 
eralized  Relevance  Eeaming  Vector  Quantization-Improved  (GREVQI)  relevance-ranked 
features.  Subsequent  classification  of  fingerprints  from  both  receivers  using  DRA  was  only 
performed  using  the  GREVQI  method  previously  discussed  in  Section  3.4.  Einally,  De¬ 
vice  Classification  was  executed  on  the  Hybrid  fingerprints.  Here,  the  term  Hybrid  is  used 
in  different  context  from  [9]  and  describes  Cross-Receiver  model  development  using  RE- 
DNA  fingerprints  derived  from  PXIe  and  USRP  collections.  As  with  the  single -receiver 
setup  described  above,  both  full-dminesional  and  quantitative  only  DRA  feature-sets  were 
used  for  Device  Classification.  Section  4.1  describes  how  the  model  was  developed  after 
dividing  the  prints  into  Training  and  Testing  sets.  Section  4.2  discusses  full-dimensional 
fingerprint  Device  Classification  using  both  Multiple  Discriminant  Analysis,  Maximum 
Eikelihood  (MDA/ME)  and  GREVQI  methods  and  compares  performance  of  PXIe  and 
USRP.  Section  4.3  details  the  DRA  process,  including  specific  quantitative  Np  selection 
and  comparison  of  qualitative  versus  quantitative  features  for  PXIe  and  USRP  receivers. 
Section  4.4  presents  Hybrid  Cross-Receiver  model  development  and  shows  MDA/ME  and 
GREVQI  Device  Classification  performance  results  for  full-dimensional  fingerprints  while 
Section  4.5  compares  quantitative  DRA  Device  Classification  results  for  MDA/ME  and 
GREVQI  models. 
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4.1  Classification  Model  Development 

The  classification  model  for  single-receiver  assessment  was  developed  using  PXIe 
and  USRP  collections  seperately.  Specifically,  a  number  of  Synchronization  Header  Re¬ 
sponse  (SHR)  from  ZigBee  emissions  from  each  receiver  were  split  into  Training  (Ns hr 
=  300)  and  Testing  (Ns hr  =  300)  for  Na  =  6  classes.  Further,  a  given  number  of  noise 
realizations  were  like-filtered  and  used  for  Monte  Carlo  simulation.  The  total  of  Nm  used 
for  Monte  Carlo  simulation  for  each  device  included: 

Nm  =  (Ns hr  =  600)  x  (N^z  =  15)  =  9000  Independent  Realizations. 

Of  this  total,  N[r  =  4500  independent  realizations  for  each  device  were  used  for 
""Training','  to  develop  the  model  via  projection  (MDA/ML)  and  with  prototype  vectors 
(GRLVQI)  as  described  in  Section  3.5.1  and  Section  3.5.2.  The  remaining  Nm  =  4500 
were  set  aside  for  ""Testing''  and  assessing  Device  Classification. 

4.2  Single  Receiver  Classification:  Full-Dimensional  (MDA/ML  and  GRLVQI) 

Both  classification  methods  were  used  for  full-dimensional  Device  Classification  where 
in  this  case,  Np  =  297.  This  was  derived  from  the  fact  that  each  SHR  was  described  by 
three  instantaneous  responses  (a  =  amplitude,  (p  =  phase,  /  =  frequency),  each  of  which 
were  in  turn  described  by  three  statistics  (cr^  =  variance,  y  =  skewness,  k  =  kurtosis). 
Further,  each  SHR  divided  into  a  fixed  Nr  =  32  subregions  across  the  entire  SHR.  The 
statistics  of  each  response  were  taken  withinin  each  subregion  resulting  in  a  total  of: 


NpuiiFeat  =  (a,  p,f)  X  (cr/  j,  k)  X  (Nr+i  =  33)  =Np  =  297.  (4.1) 

Each  RF-DNA  fingerprint  used  for  either  model  development  of  classification  thus 
contains  Np  =  291 .  Classification  was  performed  on  both  PXIe  and  USRP  using  both 
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MDA/ML  and  GRLVQI.  Fig.  4.1  shows  full-dimensional  classification  performance  for 
fingerprints  used  for  Testing.  This  was  done  for  SNR£[0  24]  dB,  keeping  in  context  of 
[30].  The  performance  of  each  device  at  each  S  NR  is  shown  as  well  as  the  cross-device  av¬ 
erage  performance  for  all  devices  noted  as  “Average.”  A  desired  benchmark  of  %C=90% 
was  set  for  easy  comparison  of  full-dimensional  and  DRA  performance.  It  can  be  seen  in 
Fig.  4.1  that  while  it  requires  a  range  of  SNRe[S.5  24]  dB  to  do  so,  each  device  as  well  as 
their  average  achieves  the  benchmark  for  both  receivers  and  both  methods. 


With  the  baseline  for  full-dimensional  classification  established,  the  remainder  of 
this  paper  will  drop  device- specific  comparison  and  provide  analysis  only  on  the  “Cross- 
Device  Average”  performance  of  all  given  devices.  Fig.  4.2  shows  the  same  classification 
performance  as  Fig.  4. 1  with  the  devices  removed,  leaving  only  the  cross-device  average  for 
each  receiver  and  method  for  comparison.  Table.  4. 1  provides  a  quick-reference  chart  for 
Fig.  4.2,  showing  the  S  NR  value  at  which  each  cross-device  average  achieves  the  arbitrary 
%C=90%  benchmark.  When  averaging  performance  for  each  individual  receiver  across 
both  methods,  it  can  be  seen  that  the  higher  end  PXIe  receiver  clearly  outperforms  the  lower 
end  USRP  receiver  by  SNR  «  4.9  dB.  Additionally,  GRLVQI,  when  averaged  across  both 
receivers,  performs  consistently  poorer,  requiring  S  NR  «  2.0  dB  gain  to  match  MDA/ML 
performance. 

4.3  Single  Receiver  Classification:  DRA  Performance  (GRLVQI) 

Due  to  the  nature  in  which  it  is  calculated,  MDA/ML  does  not  allow  for  DRA  to 
be  performed  as  it  does  not  provide  relevance-ranking  values  for  specific  features  as 
does  GRLVQI.  Refering  back  to  GRLVQI  performance  in  Table.  4.1,  PXIe  achieved  the 
%C=90%  benchmark  at  SNR  ~  12.0  dB  and  USRP  at  SNR  «  18.0  dB.  It  is  from  these 
two  S  NR  values  that  DRA  was  performed  respectively  for  each  receiver. 
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Figure  4.1:  Full-dimensional  (Nf  =  297)  elassifieation  eomparing  PXIe  and  USRP  using 


both  MDA/ML  and  GRLVQI  proeesses  for  Nci  =  6.  This  shows  all  deviee  averages  as  well 


as  the  aggregated  eross-deviee  average 


4.3.1  Quantitative  DRA  Performance. 

Quantitative  DRA  was  first  performed  in  order  to  see  how  far  of  a  reduetion  in  features 


the  model  eould  be  developed  with  before  Testing  performanee  suffered  signifigantly. 
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Figure  4.2:  Full-dimensional  (Nf  =  297)  Cross-Deviee  Averages  for  both  reeeivers  and 
both  elassifiers. 


Table  4.1:  Full-dimensional  (Np  =  297)  Cross-Deviee  Average  Benehmark  Performanee 
Comparison. 
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Signifigant  degradaded  performance  is  defined  as  a  shift  in  SNR  at  which  meeting  the 
%C=90%  benchmark  either  requires  more  gain  (positive  dB)  from  the  full-dimensional 
case,  or  where  the  benchmark  is  never  achieved  for  SNR£[0  24]  dB.  Refering  to  Section 
4.2  where  each  fingerprint  is  made  up  of  three  responses  (a,  (j),  /),  each  providing  Np 
=  99  features,  quantitative  DRA  began  with  Np  =  99.  DRA  was  subsequently  repeated 
over  all  SNR  values  as  before,  reducing  by  33%  first  to  ensure  performance  was  not 
degraded.  It  was  then  repeated  with  increased  %  reduction  each  time  until  performance 
for  either  PXIe  or  PXIe  began  to  suffer.  Overall,  quantitative  DRA  was  performed  for 
{Np  =  5, 10, 33, 66  and  99)  features.  Fig.  4.3  shows  both  receivers’  cross-device  average 
for  the  full-dimensional  feature-set  as  well  as  each  Np  DRA.  It  is  observed  that  as  in 
full-dimensional  analysis  in  Section  4.2,  high-value  PXIe  outperforms  low-value  USRP  as 
DRA  is  increased  and  Np  is  decreased.  PXIe  in  fact  consistently  achieves  the  %C=90% 
benchmark  at  the  full-dimensional  prescribed  SNR  ~  12.0  dB  for  all  except  Np  =  5.  PXIe 
does  however,  unlike  USRP,  always  achieve  the  benchmark  overall.  At  Np  =  10  features, 
USRP  performance  begins  to  suffer,  requiring  an  additional  gain  of  SNR  ~  6.0  dB.  It  then 
quickly  degrades  to  where  the  benchmark  isn’t  even  achieved  at  =  5  features. 

4.3.2  Qualitative  DRA  Performance. 

Qualitative  DRA,  describing  features  by  their  signal  responses  {a,  (f>,  f)  was  then 
performed  on  both  receivers  using  GRLVQI.  Previous  research  [9]  has  suggested  that 
unique  RF-DNA  signatures  of  ZigBee  devices  tend  to  be  easier  distinguished  when  looking 
at  the  phase  (0)  features  of  the  SHR.  Response-specific  DRA  was  performed  such  that  Np 
=  99  each  for  (a,  (j),  /).  A  model  was  created  for  each  response  feature-set  seperately.  A 
comparison  of  these  qualitative  features  as  well  as  the  quantitative  DRA  for  Np  =  99  is 
seen  in  Fig.  4.4.  Along  with  the  full-dimensional  feature  set  for  each  receiver  from  Section 
4.2,  it  is  clear  that  for  the  Atmel  ZigBee  devices  used,  (0)-only  features  far  outperform 
a  and/.  Additionally,  /-only  feature-set  for  both  receivers  performs  just  as  well  as  full- 
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PXIE 


USRP 


Figure  4.3:  Quantitative  DRA  (Nf  =  5,10,33,66,99  and  297)  Cross-Deviee  Averages 
Benehmark  Comparison 


dimensional  and  quantitative  DRA  for  Nf  =  99  features.  It  is  no  surprise  that  PXIe 
eontinues  to  outperform  USRP,  aehieving  the  %C=90%  benehmark  with  an  SNR  «  6.2 
dB  gain 

4.4  Hybrid  Cross-Receiver  Classification:  Full-Dimensional  and  Quantitative  DRA 

The  model  for  the  eross-reeeiver  setup  was  developed  in  a  similar  fashion  to  the  single- 
reeeiver  model  deseribed  in  Seetion  4.1.  In  the  Hybrid  Cross-Receiver  setup  though,  the 
model  was  developed  by  eombining  PXIe  and  USRP  fingerprints.  Again,  eaeh  reeeiver 
provided  Ns  hr  =  600  ZigBee  responses,  whieh  were  divided  into  Training  and  Testing. 
The  Hybrid  Cross-Receiver  model  though,  eontaining  fingerprints  from  both  reeeivers,  was 
split,  with  Training  =  Ns  hr  =  600  and  two  sets  (one  for  eaeh  reeeiver)  of  Testing  =  Ns  hr 
=  300  for  eaeh  of  the  Nci  =  6  elasses.  As  in  the  single-reeeiver  model,  N^z  =15  like- 
filtered,  independent  Monte  Carlo  Noise  realizations  were  added  to  eaeh  ZigBee  response 


48 


GRLVQI:  PXIE 


GRLVQI:  USRP 


Figure  4.4:  Quantitative  DRA  (Nf  =  99  &  297)  versus  Qualitative  DRA  (Nf  (a,  (p,f)  =  99) 
Cross-Device  Averages  Benchmark  Comparison. 


for  each  device  to  develop  Nir  independent  realizations  given  by: 

Nir  =  {Nshr  =  600)  X  =  15)  x  (2  Receivers)  =  18000. 

The  Hybrid  Cross-Receiver  was  then,  in  contrast  to  Section  4.1,  developed  with  Njr  =  9000 
independent  realizations  labeled  “TrainingT  The  remaining  “Testing”  fingerprints  were 
split  into  three  cases  as  summarized  in  Table.  4.2. 

In  Case  1,  the  “Testing”  fingerprints  were  a  combination  of  the  reserved  fingerprints 
for  both  receivers,  thus  Njr  =  4500  for  each  device.  Case  2  and  Case  3  tested  the  same 
Hybrid  Cross-Receiver  model  on  only  one  receiver  at  a  time.  The  reserved  Njr  =  2250  per 
device  for  PXIe  and  Njr  =  2250  for  USRP  were  used  as  “Testing”  fingerprints  for  Device 
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Table  4.2:  Case  Deseriptions  for  Hybrid  Cross-Receiver  Training  (Model  Development) 
and  Device  Classification  Testing 


Scenario 

Training  (Model) 

Testing 

Case  1 

PXIe  &  USRP 

PXIe&  USRP 

Case  2 

PXIe&  USRP 

PXIe 

Cases 

PXIe  &  USRP 

U.SRP 

Classification  in  Case  2  and  Case  3  respeetively. 

Both  Full-Dimensional  and  Quantitative  DRA  Device  Classification  were  performed 
using  both  MDA/ML  and  GRLVQI.  It  is  important  to  note  though,  that  while  DRA  was 
performed  using  MDA/ML,  the  relevanee-ranked  features  used  to  allow  this,  were  aetually 
taken  from  the  models  developed  using  GRLVQI.  Additionally,  DRA  for  both  MDA/ML 
and  GRLVQI  for  all  three  eases  shown  in  Table.  4.2,  was  performed  at  SNR  =  18.0  dB. 
This  value  was  determined  by  reealling  from  Seetion  4.3  that  the  %C=90%  benehmark  was 
aehieved  in  the  worst  ease,  for  both  reeeivers,  aiSNR  «  18.0  dB. 

4.4.1  Case  1:  Hybrid  Cross-Receiver  Testing. 

The  Hybrid  Cross-Receiver  fingerprints  used  for  Testing  were  again  a  eombination  of 
reserved,  interleaved  PXIe  and  USRP  fingerprints.  Fig.  4.5  shows  a  Full-Dimensional  {Np 
=  297)  and  Quantitative  DRA  {Np  =  5, 10, 33, 66, 99)  Device  Classification  eomparison  of 
MDA/ML  and  GRLVQI.  Table.  4.3  shows  elearly  that  for  a  Full-Dimensional  feature-set, 
MDA/ML  is  superior  to  GRLVQI,  requiring  nearly  SNR  «  2.0  dB  less  gain  to  aehieve 
%C=90%.  GRLVQI  immediately  surpasses  MDA/ML  for  Quantitative  DRA,  providing  a 
SNR  ~  3.5  dB  gain  for  Np  =  99.  Subsequent  MDA/ML  DRA  performanee  fails  to  even 
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meet  the  %C=90%  benehmark  as  GRLVQI  DRA  remains  mostly  consistent  even  as  Nf 
decreases. 


MDAML 


GRLVQI 


Figure  4.5:  Case  1:  Full-Dimensional  (Np  =  291)  and  Quantitative  DRA  (Nf  = 
5,10,33,66,99)  Device  Classification  using  Hybrid  Cross-Receiver  model  and  Hybrid 
Cross-Receiver  testing. 


Table  4.3:  Case  1:  GRLVQI  “Gain”  relative  to  MDA/ML.  “X”  indicates  an  incalculable 
value  given  the  %C=90%  benchmark  is  never  achieved  within  SNRe.[0  24]  dB. 
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4.4.2  Case  2:  PXIe  Only  Testing. 

Fingerprints  used  for  Case  2  Testing  were  entirely  from  PXIe  fingerprints  held  out 
of  model  development.  Fig.  4.6  compares  Full-Dimensional  {Nf  =  297)  and  Quantitative 
DRA  (Nf  =  5, 10,33,66,99)  Device  Classification,  again  for  MDA/ML  and  GRLVQI. 
Table.  4.4  show  that  while  MDA/ML  initially  is  marginally  better  when  running  Full- 
Dimensional  features,  GRLVQI  performance  remains  consistent  as  Nf  decreases  and  far 
outperforms  MDA/ML.  It  is  noted  also,  that  unlike  in  Case  1,  MDA/ML  performance  on 
its  own  is  much  better,  meeting  the  %C=90%  benchmark  for  all  but  Np  =  5. 
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Figure  4.6:  Case  2:  Full-Dimensional  {Np  =  297)  and  Quantitative  DRA  {Np  = 
5, 10, 33, 66, 99)  Device  Classification  using  Hybrid  Cross-Receiver  model  and  PXIe  only 
testing. 


4.4.3  Case  3:  USRP  Only  Testing. 

Case  3  Testing  fingerprints  are  comprised  of  only  USRP  fingerprints  held  out  of  model 
development.  Fig.  4.7  compares  Full-Dimensional  {Np  =  297)  and  Quantitative  DRA  {Np 
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Table  4.4:  Case  2:  GRLVQI  “Gain”  relative  to  MDA/ML.  “X”  indieates  an  inealeulable 
value  given  the  %C=90%  benehmark  is  never  aehieved  within  SNRe[0  24]  dB. 


=  5, 10, 33, 66, 99)  Device  Classification,  using  both  elassifieation  methods.  It  ean  be  seen 
from  Table.  4.5  that  the  Hybrid  Cross-Receiver  model  is  unsuitable  for  performing  Device 
Classification  with  fingerprints  only  from  the  lower  end  USRP  reeeiver.  Both  MDA/ML 
and  GRLVQI  fail  to  aehieve  the  %C=90%  benehmark  even  with  the  Full-Dimensional 
feature-set.  While  both  methods  perform  poorly,  GRLVQI  performanee  is  still  measurably 
better  as  it  stays  relatively  eonsistent  as  Np  deereases. 


Table  4.5:  Case  3:  GRLVQI  “Gain”  relative  to  MDA/ML.  “X”  indieates  an  inealeulable 
value  given  the  %C=90%  benehmark  is  never  aehieved  within  SNR&[0  24]  dB. 
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Figure  4.7:  Case  3:  Full-Dimensional  (Nf  =  297)  and  Quantitative  DRA  (Nf  = 
5, 10, 33, 66, 99)  Device  Classification  using  Hybrid  Cross-Receiver  model  and  USRP  only 
testing. 
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V.  Conclusion 


This  chapter  provides  a  summary  of  reseach  performed,  findings  and  contributions  to 
the  Radio  Frequency  Distinct  Native  Attribute  (RF-DNA)  fingeprinting  process,  and 
recommendations  for  follow-on  research. 

5.1  Summary 

ZigBee-based  networks  are  an  affordable,  widely-used  option  for  accomplishing  a 
multitide  of  tasks  within  Wireless  Personal  Area  Network  (WPAN)  applications.  Due  to 
their  low  energy  requirements,  commercial  availability,  and  low  implementation  complex¬ 
ity,  ZigBee  WPANs  are  found  in  many  buisnesses,  hospitals,  and  homes.  They  are  also 
used  in  very  important  applications  for  Industrial  Control  System  (ICS)  automation,  en¬ 
ergy  management,  and  by  the  military  for  location  and  positioning  [10].  These  sensors 
are  becoming  increasingly  exploited  and  the  systems  they  are  designed  to  protect  remain 
vulnerable  to  malicous  attacks  such  as  spoofing,  denial  of  service,  and  key  sniffing.  These 
attacks  aim  to  gain  unauthorized  access  by  spoofing  the  bit  level  credentials  required  to 
enter  a  secure  network. 

The  security  of  ZigBee  systems  must  be  increased  to  prevent  unauthorized  persons 
from  entering  networks  and  gaining  access  to  critical  information  or  control  of  critical 
systems  or  infrastructure.  One  method  for  doing  this  is  by  exploiting  known  behavior  and 
RF  fingerprint  signature  for  known  network  devices.  From  this  collection,  an  Authorized 
and  Unauthorized  list  can  be  created  that  upon  request  to  enter  a  network,  a  device  must 
submit  its  bit- level  credentials  and  be  verified  prior  to  being  granted  access.  One  method 
considered  here  is  RF-DNA  fingerprinting,  which  provides  an  added  level  of  security  by 
examining  the  actual  physical  waveform  features  emitted  by  a  device.  This  provides  a  way 
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to  single  out  a  device  and  prevent  something  as  simple  as  changing  bit  level  credentials,  or 
spoofing  a  Media  Access  Control  (MAC)  address  and  entering  a  network. 

5.2  Findings  and  Contributions 

This  research  expanded  the  capabilities  of  AFIT’s  RF-DNA  fingerprinting  process. 
Specifically,  earlier  work  in  [9]  was  expanded  with  and  knowledge  of  ZigBee  device  behav¬ 
ior  increased  by  introducting  and  investigating  RF-DNA  fingerprinting  performance  using 
two  new  receivers  and  a  new  set  of  devices.  Accurate  comparisons  between  two  different 
classification  methods.  Multiple  Discriminant  Analysis,  Maximum  Likelihood  (MDA/ML) 
and  Generalized  Relevance  Learning  Vector  Quantization-Improved  (GRLVQI),  were  made 
for  both  full-dimensional  and  Dimensional  Reduction  Analysis  (DRA)  (quantitative  and 
qualitative)  feature-sets.  Finally,  a  Hybrid  Cross-Receiver  model  for  Device  Classification 
was  introduced  and  a  first-ever  comparison  made  using  a  model  developed  with  RF-DNA 
fingerprints  from  both  a  high-value  PCI  Extension  for  Instrumentation  Express  (PXIe)  re¬ 
ceiver  and  low-value  Universal  Software  Radio  Peripheral  (USRP)  receiver;  both  receivers 
are  commercial  products  manufactured  by  National  Instruments  (NI). 

5.2.1  Single  Receiver  Assessment. 

Pull-dimensional  Device  Classification  was  performed  with  both  receivers  using  MDA/ME 
and  GRLVQI  methods.  Regardless  of  the  receiver,  MDA/ML  consistently  dominated  GRLVQI, 
with  the  latter  requiring  an  average  S  NR  ~  2.0  dB  increase  in  gain  to  match  performance. 
When  averaging  across  classification  methods  however,  the  high-end  PXIe  receiver  outper¬ 
formed  the  low-end  USRP  receiver,  requiring  SNR  ~  5.1  dB  less  to  reach  the  arbitrary 
%C=90%  benchmark. 

Quantitative  DRA  Device  Classification,  again  only  performed  using  GRLVQI,  exhib¬ 
ited  a  notable  difference  in  performance  between  PXIe  and  USRP.  While  PXIe  achieved 
%C=90%  at  SNR  ~  12.0  dB  consistently,  for  V/76[10  297],  USRP  performance  suffered 
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and  failed  to  meet  the  benchmark  for  Np  <  10.  Qualitative  DRA  using  Np  =  99  features 
proved  that  regardless  of  the  receiver  used,  the  Rfe-only  feature  set  far  outperformed  the 
Frq-on\y  and  Amp-only  feature  sets.  In  fact,  when  compared  to  quantitative  DRA  perfor¬ 
mance  using  the  top-ranked  Np  =  99  features  (combination  of  Amp,  Phz,  and  Frq  features) 
for  both  receivers. 


^^Cpp)  ~  ~  >  %C<)()Pfq  >> 

where  %CpD  is  the  average  correct  classification  using  the  full-dimensional  (FD)  feature 
set.  Finally,  on  a  comparison  of  receiver-only  performance,  regardless  of  classification 
method  or  feature-set,  high-value  PXIe  outpaced  performance  of  low-value  USRP  by 
SNR  ~  6.0  dB. 

5.2.2  Hybrid  Receiver  Assessment. 

For  “hybrid”  receiver  assessment,  the  MDA/ML  and  GRLVQI  models  were  developed 
using  fingerprints  from  both  receivers  for  both  full-dimensional  and  Quantitative  DRA 
Device  Classification.  Three  specific  cases  were  examined  where  the  Testing  fingerprint 
set  included:  1)  a  hybrid  combination  of  both  PXIe  and  USRP  fingerprints,  2)  PXIe-only 
fingerprints,  and  3)  USRP-only  fingerprints. 

Case  1:  “Hybrid  Cross-Receiver’’’’  (PXIe  and  USRP) 

When  Testing  fingerprints  were  comprised  of  both  receivers  {Hybrid  Cross- 
Receiver),  %C  performance  was  mixed.  While  both  classification  methods  met 
the  %C=90%  benchmark  for  full-dimensional  features,  MDA/ML  %C  performance 
immediately  dropped  off  for  Np  <99.  Although  MDA/ML  was  the  clear  winner  for 
full-dimensional  classification,  edging  out  GRLVQI  by  SNR  ~  2.3  dB  less  required 
gain,  subsequent  reduction  of  Np  favored  GRLVQI  by  SNR  ~  3.5  dB  less  required 
gain  than  that  of  MDA/ML  to  meet  %C=90%.  In  fact,  when  Np  <  66,  MDA/ML 
failed  to  achieve  the  %C=90%  benchmark  over  the  specified  SNR&\0  24]  dB  range. 
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Case  2:  PXIe  Only  Testing 

Utilizing  only  PXIe  fingerprints  for  Testing  greatly  increased  %C  performance  for 
both  full-dimensional  and  DRA  feature  sets  using  MDA/ML  and  GRLVQI.  While 
MDA/ML  performed  better  for  full-dimensional  classification,  when  averaged  across 
calcuable  values  (Np  =  10,33,66,99),  GRLVQI  required  SNR  «  321  dB  less  gain 
to  match  MDA/ML  %C=90%  benchmark  performance.  The  Hybrid  Cross-Receiver 
model  is  well-suited  for  use  soley  with  the  PXIe  receiver. 

Case  3:  USRP  Only  Testing 

Utilizing  only  USRP  fingerprints  for  Testing  with  full-dimensional  and  DRA  feature 
sets,  regardless  of  classification  method,  failed  to  meet  the  %C=90%  benchmark  over 
the  SNRe[0  24]  dB  range.  The  Hybrid  Cross-Receiver  model  is  deemed  unsuitable 
for  use  soley  with  the  USRP  receiver. 

Overall  when  comparing  classification  methods,  as  in  the  single -receiver  assessment, 
MDA/ML  outperformed  GRLVQI  for  a  full-dimensional  feature- set.  Averaged  among 
Case  1  and  Case  2  (Case  3  full-dimensional  analysis  failed  to  achieve  %C=90%  benchmark 
over  the  SNR&[0  24]  dB  range),  GRLVQI  required  an  additional  SNR  ~  1.6  dB  to  match 
MDA/ML  %C  performance.  Quantitative  DRA  performance  favored  GRLVQI  though, 
providing  relatively  consistent  %C  over  SNRe[0  24]  dB  for  Np  >  10.  Again,  when 
averaged  across  calcuable  values  for  Case  1  and  Case  2,  MDA/ML  required  SNR  «  3.32 
dB  additional  gain  to  match  GRLVQI  %C=90%  benchmark  performance. 

5.3  Recommendations  for  Future  Research 

This  research  has  shown  that  a  high-value  receiver  (PXIe)  generally  provides  better 
classification  performance  than  a  lower-valued  receiver  (USRP).  Further,  it  was  shown  that 
by  combining  RF-DNA  fingerprints  from  high  and  low  value  receivers,  a  Hybrid  Cross- 
Receiver  model  can  be  developed  and  when  utilized  properly,  provide  classification  per- 
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formance  results  consistent  with  models  developed  solely  from  fingerprints  for  a  single 
receiver.  This  work  is  not  complete  and  further  benefit  could  be  realized  by  investigating: 


1.  Alternative  Classifiers  -  There  are  numerous  other  classifiers  already  in  existence. 
Device  Classification  should  be  investigated  further  by  developing  models  and 
performing  classification  with  these  alternative  classifiers.  Results  varied  in  this 
research,  based  mostly  off  of  the  classification  method  used  and  the  Nf  chosen  to 
perform  classification  with.  Choosing  a  different  classifier  other  than  MDA/ML  or 
GRLVQI  may  provide  a  single  method  that  is  the  clear  winner,  outperforming  all 
other  classifiers  regardless  of  whether  Full-Dimensional  or  DRA  is  used. 

2.  Different  RF-DNA  Features  -  There  are  endless  possiblities  of  how  to  define  the 
features  that  are  used  to  describe  each  RF-DNA  fingerprint.  Further  work  should  be 
performed  to  change  the  feature  sets  that  are  used  for  classification.  Specifically 
alternatives  can  include  changing  the  size  and  number  of  subregions,  developing 
models  that  perform  quantitative  DRA  within  a  specified  time  domain  instantaneous 
signal  response  (0-only  for  example),  and  performing  fingerprint  generation  using 
different  statistical  metrics  or  combinations  thereof.  All  these  alternatives  may 
provide  a  different  set  of  features  that  can  more  consistently  meet  or  exceed 
%C=90%  at  a  lower  dB  value. 

3.  Alternative  (non-ZigBee)  Signals  -  The  ZigBee  protocol  is  a  very  small  portion  of 
the  mass  of  existing  RF  signals  capable  of  performing  device  classification  on.  The 
comparison  of  high  versus  low  value  receivers  as  well  as  creation  of  a  Hybrid  Cross- 
Receiver  may  be  found  to  be  more  suitable  if  performing  Device  Classification  on 
any  of  these  other  non-ZigBee  signals. 
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4.  Different  devices  within  the  ZigBee  protocol  -  The  Atmel  AT86RF230  is  but  one 
ZigBee  device  capable  of  being  researched  in  terms  of  its  signal  responses.  As 
the  ZigBee  protocol  is  relatively  inexpensive  and  widely  used  in  a  myriad  of 
applications,  new  devices  will  come  to  the  forefront,  and  spoofing  will  remain  a 
concern.  Investigating  alternative  devices  with  the  same  receivers  will  help  to  further 
create  a  known  pool  of  the  behavior  of  all  devices  running  the  ZigBee  protocol.  This 
information  could  be  used  to  heighten  security  within  critical  networks  that  rely  on 
ZigBee  every  day. 
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